Optical flow systems generate optical flow maps that show the relative motion of objects in a field of view (FOV) of a camera based on the change in each of a number of pixels from one frame of a captured image to the next frame. For example, a motion vector value is assigned to each of the pixels corresponding to the relative motion for that pixel. Additionally, a confidence measure may be generated for each of the motion vectors. These systems may be used in many applications including: object tracking, moving object segmentation (e.g., segmenting moving objects from still objects), and structure from motion (e.g., generating a three dimensional (3D) model of structure by taking moving images of the structure). For example, in transportation, automobiles may include optical flow systems to track moving objects within the FOV. The automobile may utilize the object tracking to, for example, determine whether a collision between the automobile and the other object, such as another vehicle, is possible, and automatically apply braking.